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( a 
Abstract © 


a, 


In this study, selected ability and achievement devices ware administered 


to a sample of 99 low achievers; the reliebilisf of vari difference 


a F 
scores was analyzed. In all cases, the reliabilities were moderately 


high. Reliabilities of differences for devices’ normed on the same popu- 
als and ‘differences for devices normed on different populations 


gvere comparable.. These results are discussed in light of current psy- 


. * 
chometric practi¢es. 


be \ 


if \ 
An Analysis of Difference Score Reliabilitjes' on Three 


Measures with a Sample of Low, Achieving oungsters. 


The U. S. Office of Education, in ite final rules and regulations 
on Proseduves for Evaluating Specific Learning Disabilities (Federal 
Register, 1977), established criteria to be used in en rere 
with specific learning disabilities. To be identified as learning dis- 
abled, team members must denongprate that hild (1) Aden not achieve 

: commensurate with his or her age and abilit evels in one or more of 
seven ayaa when provided with learning experiences ap vonrdate for the 
child's age and ability levels, and (2) the ‘child hes“a "severe discrep- 
ancy" between achievement and intellectual ability in one or more ov 
seven areas related to communication skills and mathematical abilities 
(oral siveusibn, listening comprehension, written expression, basic , 
, reading skill, reading comprehension, mathematics ‘calculation, and d 
mathematica}*reasoning). ) 

The rules and regulations do not specify the magnavade: de a dis- 
crepancy needed to be considered ‘ipeperau" Yet, diagnostic personnel 
regularly use deficit scores to identify the learning disabled. Con- 
siderable variation exists in the approaches used to identify “severe 4 

. discrepancies" (cf. volume 2, issue 4 of Learning Disability quarterly,’ 


1979). Many approaches fail to consider the reliability vg difference 


BORER) a practice strongly recommended by Salvia and Yoseldyke (1978) .. 


‘ 


~ 


ene purpose of this investigation was to ascertain’ the reliabilitids 


of su cocci scores oO ained through adnibet ration of the Woodcock-, 
. ae 

Johnson Psycho-Educationah Battery, the Revised Wechsler Intelligence | 

‘ ! - * A 


Scale for Children (WISC-R), and the Peabody Individual ‘Achievement Test’ 


re 


2 S| en 

. N * 
(PIAT). In addition, we investigated the extent to which. reliabilities 
of. differences were a function of the comparability of groups on wom y 


the tests were standardized. , 


Method 
Subjects — « 
Ninety-nine fourth’ graders fro aide different school dcteiees an 
participated in“this study. Fifty of the students fiad been labeled 


‘ XX 
"learning disabled" by their respective =— dietricrs. . Criteria used 


to identify these students differed in the different school gistricts. 
The students were low achievers, as suggested by their average total 
score on the Peabody Hratvival Achievement Test (X = 91.9, SD'= 8.78). 


| 


Forty-nine of the students wére fourth graders who had not been 
: [0 
idgntified as learniyg disabled, but who were selected as low achievers 
fe) thie basis of having scored below the 25th percentile on the Iowa 


Tests OE Basic Skills administered during the fall’ o& ‘the year in which 


these data wéxe collected. , aa? . 
. age f, c. a 
‘The ages, sex distribution, parental marieel vara: family socio- 


7 


coixinas” Shinn, and MéeCue’ (a press) 


economic status’, and i a of the two, groups were not signifi-+ 


cantly different. Yaseldyke, 


eeporees that there were ifferences of practical significamce in the 


& 


performarice of the two groups on 49 peychowstrte measures in the domains 


of intelligence, aGibesenent avin iaese functioning, self-concept, 


é 1 


and problem behavior. For purposes of this zavest tigations the two groups 


were conbings and labeled- as low petieve re: Demographic characteristics 


of the’ total group of 9d subjects are reported in Table l. soe 
a é ' 
5 Cail : : . 
{ : oe ? << 4 
: ( 
: =. . 


, 


- i? . : ar’ —_— - . —— 


r Sabie 1 about -here , oe ° 
ae ’ 4 Ty ol 


Procedure 
: soN . ‘ P eo. 

Each subject was adminjstered a battery of tests during participation 
ina larges! study. All testing was completed by qualified peychomatric+ans 
aca occurref during the same” period of time (i.e., Jarvéary) to May). Demo- 

, graphic si was —~— the parent (a) of the children and 
' a behavio ies scale(was compléted by their current teacher. / 
aan SN wv: 
Thé test battery included the Wechsler Intelligence Scale for Chil- 


dren-Revised (WISC-R) , the Peabody “Individual Achievement Test, (PIAT), ee 


selected subtests of the Stanford Achievement Test (SAT), th®] Bender Visual- 


Motor Gestalt Test (BVMGT) eee Test of Visual-Motor ‘Inte 


1 . . . 
gration (CTVMI), the Piers-Harri Self-Concept Scale, and, the Peterson- 


; 
Quay Behavior Prpblem Checkl st> Desgriptions of each of these devices, 


including information on their technical adequacy, are included in Salvia “ae 


a 


a 


and Ysseldyke (1978). The battery was selected as gne' including those 


‘ ‘ * 


devices common ly used with WD youkgoters. 


_ Additionally; selected subtdets of tne Woodcock-Johson (ws) Psycho- 


cludes 27 subtests desi ed’ to Mesure. cognitive abilities, scholastic 


or. dt 
} aptitudes, achievenent, and interests. Woodegck (1978) suggests that 


. ° 
° ‘ 


nal \ the/Battery be uged to identify students with "special problems” or disa- 
\ 9 “ 
’ das" through ‘snadyele of discrepanci 8 bétween aptitude and hieve- . 


pe 


" nient, %a (ee requiring the uge of difference scores. 


Ve: used all 12 subtéste f Part One: fests of Cognitive sia 


Educational Battery were administered,to each stident. Whe Battery in- V ; _ 


< Nv 


a 


, were analyzed. \ ms 


j ; ; a , | an 
Auditory Learning, Blending, Quantitative Caanentn: Visual Matching, 
attonyms-Synonyns, Analysis-Synthesis, Numbers Reversed, -Concept For-— 
mation, and Analogies), as be a “as seven of the 10 Tests of Achitvement 
subtests (Letter-word Identification, Word ketack, Passage Comprehension, 
Calculation, Applied Problems, Dictation, and Rroofing). No subtests = 


from Part Three (Tests of Interest Level) were administered. 


. \ 
Data Analyses all 


relations between selected "ability," “abestoge,” and "achieve~ 
ment" measures were’ obtained as were reliabilities of differences between 
“chien: Actual reliabilities for each device were available in test inanadia} 
those: for fifth grade children were used in each instance. Specifically, 


relationships between the WISC-R, WJ Broad Cognitive erueces scores, 


° and WJ Aptitude —_ stores (i.e., ability neaduires) ,and ia PIAT . 


scores and WJ Achievement cluster scores (fL.e., ‘achievement measures) 


\ 


° 


— Ne . s Results . - . 


The eeeretations between scores fromthe various peychometric le 


are ‘presented ie Table 23 the vapotead reliability for aach subtest or 


test (taken from the manuals as n parentheses after the name pf 
3 


4 . 
each device. In general, the reliabilities were accentable for“most \eypes 
s ‘ ¢ er 


of decision naking and the correlations Detyeen scores were moderately 


high. . The reliabilities of the differences betwepn scores ‘on various 
% 


devices are eeasaiked in Table 33 most of these‘are lower than either of 


the reliabilities of the tests’ from which they were derived. 


t7- =. « 


i : e “s 


= i ~_ 9 - 1 | 
\ - , ' “. ei 
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a 


= Insert Tables 2 and 3 about Here —_ . a 


‘ The ‘‘teliabilities of differences between the WJ ability and 


v 


achievement measures are quite similar and are not different from those 
‘ , o Pal 


= w 


\.. between the WISC-R ere oe WJ achievement q@ysecepancies. For the ° 
os ene clusters, it should be noted that,» 2i0 some cases, similar 

a 
subtests are ecunad together to form the different achievement eluscer 


scores. For example, Reading is comprised of ee a tacntdeicatioas 
a 


@ 


Word Attack, and Passage Compr ension gubtest scores’ and Skills is com- 
posed of Letter-Word Iddqtification, Applied Probl@ms, and Dictation | 
subtest acozes. Considerab re overlap eaiece anong the subtests 
which comprise the apsdtude clusters; Antonyms-Synonyms scores are in- a 
cluded in all aptitude clusters, Analogies scores are included in Reading _ 
and Knowledge Aptitude cluster scores, Quantitative eoncasks scores are 
3 Z included in Writtgn Language and Kasuiediga Aptitude Clustef scores, and 
( . Visual Matching ‘scores a included in Mathematics and Written Language 
- Aptitude cluster scores. Additionally, the Broad eogn itive Ability 
cluster score is comprised of all "cognitive ability" s biest scores. 


The a of differences between the PIAT achiavanént sub- a: 


‘tests and each the ability measures were also relatively similar. It . 


a. seens, from this analysis; that no distinct advantage celetive to relia~ 
bility is evidenced in differences obtained through use of the WISC-R, 
WJ Battery, and PIAT. In all case, the differences have a‘similar 

= _ dagree of rebiability; this in no way suggeste thet the use of those 


differences is a valid Or rec nded diagnostic technique. 4 oe 


fer, 


\ _ | 
; e. . J scussion | 
fa <. x a } 
: ) “A common and often rec endet practice for decision making in: the J 


¢ 
field act learning désabilities is the identification of discrepancies 


ss < “Aa "4 . 


ar 4 


‘<) 
ERIC 


\c 


{ f The 


within e child's test performance. importance of this practice ia 


ee by Faguier ions. pressnt in Popite: Tay 94-142 (Education for 


All Handicapped Children Act). This ‘research was designed to evaluate 


sthe- reliability of difference (i.e., discrepancy) scores for selected 


~* 


psychometric davicas administered to a sample of underachieving youngsters. 
An analysis of the results indicated that comparable statistics 
obtained when. different devices were administered. In all cases, thf 
reliability if ditincwics scores was moderately high (e.g., range = 0.52 
to 0:73): Similarly, differences in these reliabilities did not seem to 
be a function of the devices®on-whtch they ware based. | . 
— Pe é 
In ‘discussing the use of "profile aiaivatel and (analysis of differ- 
ence scores," Salvia and Yeseldyke (1978) suggested that these tasks are 
difficult because,typically the tests used be aatine ability-achievement 
discrepageies are. normed on different sanbieé and correlations between 
them generally are not ,available. The problems inherent in decision~ | 
making devices normed on different samples have not been addressed by 


‘ 
swe have opvained correlationd between tests often 


used in plaking decisions aboutgAidiqcrepancies.” The reliabilipstes of 


differences obtained with tes®@i*of. different norms (e.g., WISC-R and c. 

PIAT) werd similar to those obtained or a devices with one normative ' 

population (e.} ext nt, to which these data are useful to > 
»! 


decision makey&.remains the subject of bo h applied an asic research. 


= 
( 


a 7 
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ce bled 5 


ee a Characteristics of 99 Low Achievers ‘ 


Child's “ Clifld's = = .-.. Parentat ee a 7 2 
Sex Age (months) === Marital Status. _ Family SES Family Income 


ere ae | ee nee - 
Male Female . X ° SD' . Married Unmarried “X.- Sic» X . SD 


ise ‘ fe re i. Z Lo : 7 . 
Th 121,50" 4,52. 4 50,91 23.91 $21,138, $10,752 


¢ ; 
4, a 
6 voc 
' r : : 14. | 4 
~t 4 i i i 


‘i ¥ e F ' | - an : 
rbtee A Z fe q 
. u ‘ ‘ : . , * “’ 
q rm | | Correlations between Selected Psychometric Devices ON 
; oe by ; ee a re ‘ 
: pe a 8 
. Achievement Scores - | 
_ Ability | - lwo w® 4. {Weltten PY. 
Scores +, Spelling Reading [Mathematics | Language | killa’ 
4 (.53) (.95) | (.92) “1 (.94) | (95) 
 WISC-R a 
Pull Seale 19." | 7 
CS) Cr ee a 
 WISCR | ‘7 vy 
Verbat 1Q : i a 
en!) | 
" WISC-R | # 
Performance 1Q. I 
_ (89) , 
wi | : 


. Broad Cognitive | 
Cluster Score*, 
(96) | 
WI J 
Math Aptitnde: 
Cluster Seope 
(.86)’ 


Readjng Aptitudé 
Cluster Score. 
(93) 

f ‘ 
Wl ‘ 
Writing Aptitude 
f Cluster Score 


i (90) 
*f 
j Wd 
Knowledge Ap- 
4 titude 
Cluster Score 
(.93), 


Note. Numbers in parentheses aré reliabilities for each device, a. 
\~ Number of subjects = 99, - ce a ge i. | 
eee oe ee ee 


eo oc sy ; _ pe : ; ig hee? 
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Be ge Sd mee et Table Ss , # e | .- 
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ry oa ee of DiEfgelace Stores for, Selected Paychonetyic Devices 
oe ' i 5 
. ‘ f "4 . \ : 
s, { ~ : 5 oe 
| , -+ Achievement Scores ~ . - foe, s 
‘Ability, , °° [PIATD [PIT | RIAT’ 
Scores, 1 Spelling 
\ 
“WISC-R 


. WISC-R 


WISCER 


Broad Cognitive 


+ GB), 


f- 


. 
’ 


a’ 


~ \ 


: Hath Aptitude 


Full Scale 1Q 
(,95) | 
) " g 
Verbal IQ 

(93) 


Performance 1Q. 
(89) 
WJ 


‘Cluster Score — 
(.96) - . 


WJ 


Cluster Score - 


Wo, 

Reading Aptitude 

Fister Score 
(.93). 


Ww 

Writing Aptitude 

Cluster Score © 
(.90) | 


Wi ; mt a a ; . 
Knowledge Apti- Pe . "ge oe ~ & Pale a i 2 
tude = apa) BG SS 64 66 16. | .89 -80 87 | Bh 
Cluster Score ~| "am, at | : 

.93) : ‘se a - . ; a 
Note. Numbers in parentheses are Padi lities for each device (taken from the respective test manuals), 
Number of subjects = 99. 
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